Dynamic memory allocation refers to the allocation of memory to a computer program during the runtime of that program. The dynamic allocation helps efficiently distribute limited memory resources to several programs that are executing at the same time. Memory may be allocated among said programs from a pool of unused memory managed by a data structure called the heap or other suitable memory management data structure. When a program no longer needs the memory, the allocated memory is released back (i.e., is deallocated) to the heap.
Dynamic memory allocation in a virtualized environment is generally more challenging because the memory resources are shared among several virtualized components such as a hypervisor, one or more virtual machines (VMs), and the corresponding guest software. A hypervisor runs on a host machine to support the allocation of resources to the VMs. A VM allows for guest software to be executed on the host machine without the guest software being aware of the underlying hardware specification of the host machine. In other words, a VM provides a platform independent (i.e., a virtual) execution environment for the guest software.
In a virtualized environment, the memory space for the guest software is divided into blocks, typically 4 KB, called pages. The physical memory (i.e., the host machine's memory) is also divided into blocks, also typically 4 KB. When the host's memory is full, the data for virtual pages that are not present in host's memory are stored on disk. Therefore, three memory layers may be present: guest virtual memory, guest physical memory, and host physical memory. When the memory is being dynamically allocated, each guest consumes memory based on its configured size, plus additional overhead memory for virtualization.
The guests sometimes use more memory than the host has physically available. For example, three guests with 2 GB memory allocated to each may run on a host with 4 GB of physical memory. In that case, the memory is said to be overcommitted for about 2 GB. To improve memory utilization, the hypervisor may deallocated memory from idle guest software and allocate the memory to guest software that need more memory.
When the total amount of memory in the host becomes low, none of the guest software may free guest physical memory because the guest OS cannot detect the host's memory shortage. Particularly, Java Virtual Machines (JVMs), which are platform-independent execution environments for converting and executing Java bytecode, lack an automatic mechanism to externally control the heap size during run-time. In a JVM, minimum and maximum values for the heap size are defined when the JVM starts. During run-time, the heap size is usually increased and may not be reduced by the JVM.
In the above scenario, the OS assigns a portion of the physical memory to the JVM as it does to any guest process. If the JVM consumes more memory than the physical memory available, the OS starts swapping user-space memory to disk or similar external devices. This process significantly reduces the performance of the JVM and the overall system. In some implementations, when multiple VMs run over a single host and share a fixed amount of physical memory, the hypervisor may over-commit memory to reduce costs, by giving the VMs the illusion that they have more memory than is actually available on the physical machine.
Due to the fact that there is no cooperation between the JVM and the hypervisor, when a VM is running a JVM inside and the JVM's heap size is larger than the physical memory assigned by the hypervisor, swapping will occur at the host or the guest level, also reducing the performance of the VM and the overall host. There is known trade-off between the JVM performance and the heap size. Even if reducing the heap size decreases the JVM performance, this performance degradation is an order of magnitude less than the performance degradation caused when the heap size is larger than the physical memory available.